The present invention relates to an image processing apparatus and storage medium storing an image processing program.
FIG. 22 is a view showing a representative electronic still camera system. Image data obtained by photographing an object with an electronic still camera 804 shown in (a) of FIG. 22 is normally stored in a memory card 805 shown in (b) of FIG. 22. When a color printer 801 shown in (c) of FIG. 22 is connected via a connection cable, a color image can be printed on a medium of a size as small as about A6.
The memòry card 805 stored in a predetermined adapter can be inserted into a docking station 802 shown in (d) of FIG. 22. An image can be observed on a TV monitor 800 shown in (e) of FIG. 22 through the docking station 802. When an MO drive 803 shown in (f) of FIG. 22 is connected to the docking station 802, image data can be stored in an MO disk 806 shown in (g) of FIG. 22.
Image data obtained by the electronic still camera 804 can be transferred to a desktop personal computer 809 shown in (h) of FIG. 22 through a connection cable. When the memory card 805 is stored in a predetermined adapter, image data can be loaded into a notebook personal computer 810. In addition, image data in the MO disk 806 can be transferred to the notebook personal computer 810 through a predetermined MO drive. The monitor of the desktop personal computer 809 or the liquid crystal screen of the notebook personal computer 810 is capable of more precise display than the TV monitor 800. An image can be printed by connecting a color printer 811 shown in (j) of FIG. 22, which is larger than the color printer 801, to the desktop personal computer 809 or notebook personal computer 810 via a connection cable.
In the above electronic still camera system, the number of pixels of the electronic still camera is generally about 640xc3x97480 (about 300,000 pixels) to 1,280xc3x971,024 (about 1,300,000 pixels). A TV monitor requires about 300,000 pixels, the monitor of a personal computer requires about 1,000,000 pixels, printing at 300 dpi on A6-sized paper requires about 1,300,000 pixels, and printing on A4-sized paper requires about 5,000,000 pixels. Even in the electronic still camera, the number of pixels relatively decreases upon digital zoom or photographing in a size xc2xdxc3x97xc2xd the number of pixels in accordance with the image quality mode. In the entire system, the number of pixels for input does not match that required for output in many cases.
Such an electronic still camera generally uses an imaging system using a one CCD, two CCD, or three CCD with spatial pixel offset. As a technique of improving resolution by spatial pixel offset, a general description is given in, e.g., Yuji Kiuchi, ed., xe2x80x9cHandbook of Image Input Techniquexe2x80x9d, 1st Ed., Nikkan Kogyo Shimbun, Mar. 31. 1992, pp. 143-145 and pp. 259-260.
In this imaging system, one pixel is comprised of a plurality of color signals, and at least one color signal is often missed in accordance with the pixel position.
FIG. 23 shows the layout of complementary color mosaic filters of cyan (Cy), magenta (Mg), yellow (Ye), and green (G) generally used in a one CCD imaging system. Referring to FIG. 23, for the nth line and (n+1)th line of an even field, luminance signals are represented by Ye,n and Ye,n+1, respectively, and color difference signals are represented by Ce,n and Ce,n+1, respectively. For the nth line and (n+1)th line of an odd field, luminance signals are represented by Yo,n and Yo,n+1, respectively, and color difference signals are represented by Co,n and Co,n+1, respectively. These signals are given by
Yo,n=Yo,n+1=Ye,n=Ye,n+1=2R+3G+2Bxe2x80x83xe2x80x83(1)
Co,n=Ce,n=2Rxe2x88x92Gxe2x80x83xe2x80x83(2)
Co,n+1=Ce,n+1=2Bxe2x88x92Gxe2x80x83xe2x80x83(3)
where Cy, Mg, and Ye are represented, using G, red (R), and blue (B), by
Cy=G+Bxe2x80x83xe2x80x83(4)
Mg=R+Bxe2x80x83xe2x80x83(5)
Ye=R+Gxe2x80x83xe2x80x83(6)
As is represented by equation (1), luminance signals are generated in correspondence with all lines of even and odd fields. However, two color difference signals are generated only every other line, and each missing line is compensated by linear interpolation. After this, matrix calculation is performed to obtain three primary colors of R, G, and B. In this method, the color difference signal has an information amount only xc2xd that of the luminance signal, so an artifact called color moire is generated at an edge portion. Generally, to reduce color moire, a low-pass filter using a quartz filter is arranged on the front side of the imaging element. However, when the low-pass filter is inserted, the resolution becomes low.
Instead of simple interpolation using only the color difference signal, methods of correcting the color difference signal using the luminance signal component have been proposed. As one method, a luminance signal Y is prepared by linear interpolation. A color difference signal C is compensated by linear interpolation in a region where the change amount of the luminance signal Y is small. In a region where the change amount is large, the luminance signal Y is rearranged as
Cxe2x80x2=aY+bxe2x80x83xe2x80x83(7)
where a and b are constants to obtain a restored color difference signal Cxe2x80x2.
In a technique disclosed in Jpn. Pat. Appln. KOKAI Publication No. 5-56446, the luminance signal Y is prepared by linear interpolation. For the color difference signal C, the luminance signal Y and color difference signal C are processed by a low-pass filter constructed by an electrical circuit to obtain their low-frequency components Ylow and Clow. The color difference signal Cxe2x80x2 in which missing information is restored can be obtained by                               C          xe2x80x2                =                  Y          ⁢                      xe2x80x83                    ⁢                                    C              low                                      Y              low                                                          (8)            
This amounts to replacement of the color difference signal with a corrected luminance signal. In the above prior art, the color difference signal is corrected with reference to the luminance signal, though the luminance signal has an information amount only xc2xd that of the three CCD imaging system. In these techniques as well, a low-pass filter using a quartz filter must be used to reduce color moirxc3xa9. For this reason, the resolution of the luminance signal serving as a reference further lowers, and an image quality equivalent to that of the three CCD imaging system cannot be realized.
As described above, in the prior art, a color difference signal is compensated by linear interpolation or on the basis of a luminance signal, and a missing color signal cannot be accurately restored at a high speed. Under the circumstance, the present invention has as its object to provide an image processing apparatus capable of accurately restoring a missing color signal at a high speed.
In the prior art, a luminance signal or color difference signal is generated by simple addition/subtraction in units of lines independently of edges or color boundaries in an image. Hence, false colors generated at edges or color boundaries cannot be reduced without sacrificing resolution. Under the circumstance, the present invention has as another object to provide an image processing apparatus capable of reducing false colors generated at edges or color boundaries without decreasing resolution.
In the prior art, a signal is processed without considering the relationship between the number of pixels of the imaging system and that of the output system, and therefore, an appropriate image quality cannot be obtained in an appropriate processing time. Under the circumstance, the present invention has as still another object to provide an image processing apparatus capable of obtaining an appropriate image quality in an appropriate processing time.
In the prior art, a signal is processed without considering the relationship between the number of pixels of the imaging system and that of the output system, and therefore, an appropriate image quality cannot be obtained by automatic processing in an appropriate processing time. Under the circumstance, the present invention has as still another object to provide an image processing apparatus capable of obtaining an appropriate image quality by automatic processing in an appropriate processing time.
In the prior art, a signal is processed without considering the relationship between the number of pixels of the imaging system and that of the output system, and therefore, priority cannot be given to one of the processing time and image quality which the user chooses. Under the circumstance, the present invention has as still another object to provide an image processing apparatus capable of processing a signal while giving priority to one of the processing time and image quality which the user chooses.
In the prior art, a color difference signal is compensated by linear interpolation or on the basis of a luminance signal, and therefore, a missing color signal cannot be accurately restored. Additionally, a signal obtained by compensating for a missing color signal once by linear interpolation or on the basis of a luminance signal cannot be processed again and accurately restored. Under the circumstance, the present invention has as still another object to provide an image processing apparatus capable of accurately restoring a color signal even after it is processed by linear interpolation or the like.
In order to achieve the above objects, according to the first aspect of the present invention, there is provided an image processing apparatus having a one CCD, two CCD, or three CCD with spatial pixel offset imaging system, comprising:
a parameter calculation section for sequentially scanning an image signal in units of pixels and calculating a parameter for region segmentation from at least one neighboring region containing a current pixel of interest;
an image signal segmentation section for segmenting the image signal into uniform regions having single color correlation on the basis of parameters calculated by the parameter calculation section;
a regression section for regressing, to a linear formula, the color correlation between color signals present in the uniform region segmented by the image signal segmentation section; and
a first restoring section for restoring a missing color signal on the basis of the linear formula and the color signals present in the uniform region segmented by the image signal segmentation section.
According to the second aspect of the present invention, there is provided an image processing apparatus having a one CCD, two CCD, or three CCD with spatial pixel offset imaging system, comprising:
a local region extraction section for sequentially scanning an image signal in units of pixels and extracting a local region containing a current pixel of interest;
a parameter calculation section for setting a plurality of small regions in the local region extracted by the local region extraction section and calculating a parameter for region segmentation from each small region;
a local region segmentation section for segmenting the local region into uniform regions having single color correlation on the basis of parameters calculated by the parameter calculation section;
a selective regression section for selecting color signals belonging to the same region as that of the current pixel of interest in the local region segmented by the local region segmentation section on the basis of the uniform region and regressing color correlation between the color signals to a linear formula; and
a first restoring section for selecting color signals belonging to the same region as that of the current pixel of interest in the local region segmented by the local region segmentation section on the basis of the uniform region and restoring a missing color signal in the same region as that of the current pixel of interest on the basis of the color signals and the linear formula.
According to the third aspect of the present invention, there is provided an image processing apparatus having a one CCD, two CCD, or three CCD with spatial pixel offset imaging system, comprising:
a first restoring section for restoring a missing color signal of an image signal sensed by the imaging system by linear interpolation;
a conversion section for converting the image signal restored by the first restoring section into an original image signal obtained by the imaging system;
a second restoring section for restoring a missing color signal of the image signal converted by the conversion section on the basis of color correlation between color signals; and
a switching section for switching between the conversion section and the second restoring section.
According to the fourth aspect of the present invention, there is provided a computer-readable storage medium which stores a program comprising an instruction causing a computer to execute:
parameter calculation processing of sequentially scanning, in units of pixels, an image signal obtained by imaging with a one CCD, two CCD, or three CCD with spatial pixel offset imaging system and calculating a parameter for region segmentation from at least one neighboring region containing a current pixel of interest;
image signal segmentation processing of segmenting the image signal into uniform regions having single color correlation on the basis of calculated parameters;
regression processing of regressing, to a linear formula, the color correlation between color signals in the uniform region; and
restoring processing of restoring a missing color signal on the basis of the linear formula and the color signals present in the uniform region.
According to the fifth aspect of the present invention, there is provided a computer-readable storage medium which stores a program comprising an instruction causing a computer to execute:
local region extraction processing of sequentially scanning, in units of pixels, an image signal obtained by imaging with a one CCD, two CCD, or three CCD with spatial pixel offset imaging system and extracting a local region containing a current pixel of interest;
parameter calculation processing of setting a plurality of small regions in the extracted local region and calculating a parameter for region segmentation from each small region;
local region segmentation processing of segmenting the local region into uniform regions having single color correlation on the basis of calculated parameters;
selective regression processing of selecting color signals belonging to the same region as that of the current pixel of interest in the local region on the basis of the uniform region and regressing color correlation between the color signals to a linear formula; and
selective restoring processing of selecting color signals belonging to the same region as that of the current pixel of interest in the local region on the basis of the uniform region and restoring a missing color signal in the same region as that of the current pixel of interest on the basis of the color signals and the linear formula.
According to the sixth aspect of the present invention, there is provided a computer-readable storage medium which stores a program comprising an instruction causing a computer to execute:
first restoring processing of restoring a missing color signal of an image signal obtained by imaging with a one CCD, two CCD, or three CCD with spatial pixel offset imaging system by linear interpolation;
conversion processing of converting the image signal restored by the first restoring processing into an original image signal obtained by the imaging system;
second restoring processing of restoring a missing color signal of the converted image signal on the basis of color correlation between color signals; and
switching processing of switching between the conversion processing and the second restoring processing.
Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.